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Abstract ofWO03036460 

An information storage medium including a 
markup document and AV data, a recording 
method, a reproducing method, and a 
reproducing apparatus therefor are provided. The 
information storage medium includes AV data 
including audio data and video data, a markup 
document, and scene synthesis information 
which describes one of at least two display 
modes for displaying a markup document scene 
obtained from the markup document and an AV 
scene obtained from the AV data together. 
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(57) Abstract: An information storage medium including 
a markup document and AV data, a recording method, 
a reproducing method, and a reproducing apparatus 
therefor are provided. The information storage medium 
includes AV data including audio data and video data, 
a markup document, and scene synthesis information 
which describes one of at least two display modes for 
displaying a markup document scene obtained from the 
markup document and an AV scene obtained from the AV 
data together. 
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INFORMATION STORAGE MEDIUM INCLUDING MARKUP— 

DOCUMENT AND AV DATA, RECORDING METHOD, REPRODUCING 
METHOD, AND REPRODUCING APPARATUS THEREFOR 

5 Technical Field 

The present invention relates to a method for displaying AV data 
together with a markup document, and more particularly, to an 
information storage medium including a markup document and AV data 
so that the markup document and the AV data can be reproduced and 
10 displayed together in various ways in an interactive mode, a recording 
method, a reproducing method, and a reproducing apparatus therefor. 

Background Art 

Digital video discs (DVDs), (hereinafter, referred to as "interactive 
15 DVDs") on which a markup document is recorded together with AV data, 
have been initially developed for movie applications and later started to 
be widely used in a computer industry. AV data recorded on interactive 
DVDs can be reproduced in two ways: a video mode displayed 
_ identically in a conventional DVD, and an interactive mode in which 
20 reproduced AV data is displayed in a display window defined by a 
markup document. If the interactive mode is selected by a user, a 
viewer installed in a DVD reproducing apparatus displays a markup 
document recorded on an interactive DVD. AV data selected by the 
user is displayed in the display window of the markup document. For 
25 example, when the AV data is a movie, the movie is run in the display 
window of the markup document, • and various additional information 
including scenario, history, and actors' pictures related to the movie are 
displayed in the remaining part of a screen excluding the display window 
of the markup document. The additional information includes image 
30 files or text files. 
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recorded in the markup document, and more preferably, the scene 

synthesis information includes a cascading style sheet (CSS) inserted in 
the link tag. 

The CSS includes at least one of display mode designation 
5 information for designating a display mode of the AV scene, an AV 
trimming area designation information for designating an area to 
increase and reduce a desired portion of the AV scene, background 
color designation information for designating a background color of the 
AV scene, trimming area designation information for designating a 

1 0 trimming area of a scene in which the AV scene is synthesized with the 
markup document, window designation information for designating a 
window in which a scene where the AV scene is synthesized with the 
markup document is displayed on the screen of a display, screen display 
type information representing an aspect ratio of a scene in which the AV 

15 scene is synthesized with the markup document scene, and window 
designation information of the AV scene for designating an area of a 
window in which a trimmed AV scene is displayed on the screen of the 
display. 

Further, the scene synthesis information further includes an object 
20 having property variables for controlling the CSS and a program for 
controlling the CSS on the basis of the object. 

To achieve the above aspects, according to another aspect of the 
present invention, there is provided a method for recording AV data 
including audio data and video data on an information storage medium. 
25 The method comprises (a) recording the AV data, (b) recording a markup 
document to be displayed together with the AV data, and (c) recording 
scene synthesis information which describes one of at least two display 
modes for displaying a markup document scene obtained from the 
markup document and an AV scene obtained from the AV data together. 
30 In step (a), the AV data is recorded in a video directory, in step (b), 

the markup document is recorded in an interactive directory, and in step 
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intonxiatiDa^ispJavinQ an AV scene obtained by reproducing the AV 

data and a markup document scene obtained by reproducing the markup 
document to be displayed together with the AV data, and changing the 
output state of the markup document scene in response to scene mode 
5 change. 

To achieve the above aspects, according to another aspect of the 
present invention, there is provided an apparatus for reproducing AV 
data including audio data and video data recorded on an information 
storage medium. The apparatus includes a reading unit which reads 

10 the AV data and a markup document to be displayed together with the 
AV data, a decoder which decodes the AV data read by the reading unit 
and outputs an AV scene, and a controller which interprets the markup 
document read by the reading unit, outputs a markup document scene, 
interprets scene synthesis information which describes at least two 

15 display modes for displaying the markup document scene together the 
AV scene, and displays the AV scene and the markup document scene 
in one of the display modes according to the interpreted scene synthesis 
information. 

To achieve the above aspects, according to another aspect of the 
20 present invention, there is provided an apparatus for reproducing AV 
data including audio data and video data recorded on an information 
storage medium. The apparatus includes a reading unit which reads 
the AV data and a markup document to be displayed together with the 
AV data, a decoder which decodes the AV data, read by the reading unit 
25 and outputs an AV scene, and a controller which interprets scene 
synthesis information corresponding to a scene mode set in a 
reproducing apparatus or set by a user, interprets the markup document 
to be displayed together with the AV data read by the reading unit using 
the interpreted scene synthesis information, displays a markup 
30 document scene, and changes the output state of the markup document 
scene in response to scene mode change. 
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FIG. 13 shows a reference diagram illustrating scenes in which 

AV data for 16 x 9 is displayed on a display for 4 x 3 in a letter box 
shape and a pan & scan shape; 

FIG. 14 shows a reference diagram illustrating a markup 
5 document using the concept of a hot area; 

FIG. 15 shows variations in coordinate systems needed in 
mapping the area selected using viewport: to the area designated using 
window:; 

FIG. 16 shows a flowchart of another preferred embodiment of a 
10 reproducing method according to the present invention 

FIG. 17 shows a reference diagram in which AV data for 16 x 9 is 
synthesized with a markup document for 4 x 3 in embedded, background, 
and PIP modes, respectively, 

FIG. 18 shows a reference diagram in which the AV data for 16 x 
15 9 is synthesized with the markup document for 4 x 3 in a background 
mode, and then is displayed on a display for 16 x 9; 

FIG. 19 shows a reference diagram illustrating a display, a 
markup document, and AV data according to different aspect ratios; 

FIG. 20 shows a reference diagram illustrating the ratio of width to 
20 length in pixels in a display for 16 x 9 and the ratio of width to length in 
pixels in a display for 4x3; 

FIG. 21 shows a reference diagram illustrating various aspect 
ratio conversion; 

FIG. 22 shows a reference diagram in which AV data for 16 x 9 
25 and a markup document for 4 x 3 each having a window area are 
synthesized with each other, and then is displayed on a display for 16 x 
9; and 

FIG. 23 shows a flowchart of another preferred embodiment of a 
reproducing method according to the present invention. 

30 

Best mode for carrying out the Invention 
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through an application program interface (API), and a client 

interpretation engine. The API is a predetermined special method for 
requesting processing of an operation system or another application. 
The client interpretation engine is implemented with a JavaScript or Java 
5 interpretation engine, interprets a program coded with JavaScript or Java, 
like a web browser and executes the program. Furthermore, the 
presentation engine 3 may further include a Plug-In. The Plug-In 
enables files in various formats included in the markup document or 
called by the markup document, to open. The presentation engine 3 

10 interprets the scene synthesis information according to the present 
invention and transmits a decoding command and a display command 
according to the result of interpretation to the AV decoder 2 and the 
blender 4, respectively. The presentation engine 3 also provides the 
markup document scene to the blender 4. The AV decoder 2 decodes 

15 the AV data according to the provided decoding command and outputs 
the decoded AV data to the blender 4. The blender 4 displays the 
markup document scene and the AV scene together according to the 
display command. 

In order to perform a reproducing method according to an 

20 embodiment of the present invention, the presentation engine 3 calls a 
style sheet linked to or embedded in the markup document manufactured 
by the reading unit 1 and interprets the style sheet. Information on a 
method for synthesizing (displaying) the markup document scene and 
the AV scene is described in the style sheet. 

25 In order to perform the reproducing method according to another 

embodiment of the present invention, the presentation engine 3 
interprets a screen mode (aspect ratio, resolution, and video output 
mode) set in the reproducing apparatus or set by the user and outputs a 
command according to the set screen mode using scene synthesis 

30 information which is defined using a default style sheet set in the 
reproducing apparatus or a style sheet provided by a manufacturer (in 
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interprets viewport and window (document-viewport:, document-window :) 
only for the markup document defined in the scene synthesis information, 
respectively, and viewport and window (video-viewport:, video-window:) 
only for the AV data. In order to perform the reproducing method 

5 according to yet still another embodiment of the present invention, unlike 
the AV decoder 2 having a switching unit, even though the display is set 
to the 4 x 3 size, in the embedded mode or PIP mode, the AV decoder 2 
having no additional switching unit outputs the AV stream for 16 x 9, in 
such a way that the display is set to the 16x9 size, without outputting 

10 the AV stream in the letter box or pan & scan shape according to the 
viewport and window only for the interpreted markup document and the 
viewport and window only for the AV data. 

FIG. 2 is a reference diagram illustrating a scene alignment order 
(Z-order) of a display (not shown) connected to the reproducing 

15 apparatus of FIG. 1. Referring to FIG. 2, a final scene shown to the 
user is generated by four scenes added physically. A scene 21 on 
which a pointer is displayed, is placed foremost, followed by a markup 
document scene 22 according to the markup document, an AV scene 23 
according to the AV data, and a background scene 24 generally 

20 displayed in a single color. 

FIG. 3 shows the structures of files of the DVD 100 of FIG. 1 
according to a preferred embodiment of the present invention. 
Referring to FIG. 3, a video directory VIDECMTS including AV data, and 
an interactive directory DVD_ENAV including data for supporting an 

25 interactive function, such as a markup document, are provided in a root 
directory. AV data and reproduction control information (also, known as 
navigation data) are recorded in the video directory VIDEO_TS. The 
reproduction control information includes information referred to, so as to 
decode the AV data. The markup document and the scene synthesis 

30 information are recorded in the interactive directory DVD_ENAV. 

11 
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snene,._as. descri bed with reference to FIG. 2. The PIP mode enables 

the AV scene to seem to be output on the markup document scene from 
a user's viewpoint. Preferably, the PIP mode is subdivided by the 
placement location and size of the AV scene. In the present 

5 embodiment, the PIP mode is divided into several shapes using PIP-# 
such as PIP-1 and PIP-2. In the PIP mode, the user can change the 
location and size of the AV scene using a remote controller (not shown). 
Meanwhile, an example of an XML code for constituting the markup 
document A. HTM shown in FIG. 4 is as follows. 



<?xml version="1 .0"?> 
<!DOCTYPE html 

PUBLIC "-//DVD//DTD XHTML DVD HTML 1 .0//EN" 

M http://wvw.dvdforum/dvdenav/dvdhtml-1-0.dtd"> 
<html> 

<head> 

<title>DVD HTML PIP mode sample</title> 
<link rel="stylesheet" type="text/css" href= M a.css"> 
</head> 

. <body onload="dvdvideo.playO;"> 

<table border= M 0" width="720 H height="480"> 
<tr> 
<td> 

<img src= ,, a.png" width="1 00%" height="100%" border="07> 
</td> 
</tr> 
</table> 
_ </body> 
</html> 

10 A. HTM 



It is apparent that a style sheet file A.CSS is linked to the above 
source code using a link tag. An example of a style sheet source code 
is as follows. 



©screen-display 
{ 

video-placement: pip 

background-color: #00000000 

viewport : rect(0px,719px l 479px,0px) 

window: rect(0px,719px,479px,0px) 

video-viewport: rect(0px,71 9px,479px,0px) 

} 

15 A. CSS 
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<tr> 




<td width="277"Tieigm^ , 296"> " 




<img src="b2.png" width="277" height="296" border="07> 




</td> 




</tr> 




</table> 




</body> 




</html> 





B.HTM 



©screen-display 
{ 

video-placement: embedded 
background-color: #00000000 
viewport: rect(0px,719px f 479px l 0px) 
window: rect(0px,71 9px,479px,0px) 
video-viewport: rect(0px,71 9px,479px,0px) 

} 

B.CSS 

A display window in which the AV scene is to be displayed using 
an object tag, is defined in B.HTM. That is, the area of the display 
5 window is determined by properties such as "width" and "height" in the 
"object" tag. In addition, it is apparent that a style sheet file B.CSS is 
linked to B.HTM using a link tag. Here, B.CSS means a cascading style 
sheet (CSS) file. CSS enables "style" of the markup document to be 
described. Alternatively, the style sheet file can be lined to B.HTM 

1 0 using a style tag. 

FIG. 6 shows a background mode, as one display mode according 
to the present embodiment. Referring to FIG. 6, in the background 
mode, the markup document scene is output on the AV scene. An 
example of an XML code for constituting the markup document C.HTM 

15 shown in FIG. 6 and an example of a style sheet source code are as 
follows. Similarly, a style sheet file C.CSS is linked to C.HTM using the 
link tag. 

<?xml version="1 .0"?> " — 

<!DOCTYPE html 

PUBLIC "-//DVD//DTD XHTML DVD HTML 1 .0//EN" 

M http://www.dvdforum/dvdenav/dvdhtml-1-0.dtd M > 
<html> 

<head> 

<title>DVD HTML Background sample</titte> 
<link rel="styiesheet" type="text/css" href="c.css"> 
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="dvd:"> represents a background in a window defined to out put the 
markup document scene. That is, in the background mode according to 
the present embodiment, the AV scene is displayed in the entire scene 
but if the background using the body tag is designated, the AV scene is 
displayed only in a predetermined window scene. 

2. background-color: It designates a background color of a 
scene formed of a single color. A value is <color>, and an initial value 
may vary according to a user agent (UA). 

3. window: It designates a window in which a scene where 

the AV scene is synthesized with the markup document scene, is 
displayed. A value is <shape>, and an initial value is rect (0%, 100%, 
100%, 0%). 

4. viewport: It designates a trimming area of the scene in which 
the AV scene is synthesized with the markup document scene. A value 
is <shape>, and an initial value is rect (0%, 100%, 100%, 0%). 

5. video-viewport: It designates an area to increase and reduce 
a desired portion of the AV scene. A value is <shape>, and an initial 
value is rect (0%, 100%, 100%, 0%). Here, the value of the defined 
<shape> is rectangle (<top> <right>,<bottom>,<left>). 

FIG. 7 shows a reference diagram more specifically illustrating a 
display mode of the AV scene and the markup document scene 
according to video-viewport: and video-placement:. Referring to FIG. 7, 
if an area a for increasing and reducing from the AV scene is designated 
by video-viewport:, when video-placement: is a background mode, a 
trimming area a is displayed in a background scene, and a markup 
document scene (not shown) is displayed in the trimming area a. When 
video-placement: is a PIP mode, the trimming area a is displayed in a 
designated location. When video-placement: is an embedded mode, 
the trimming area a is embedded in a display window defined by a 
markup document and is displayed in the display window. FIG. 7 shows 
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A property variable for designatin g a dis play mrvte g propeAy, 
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variable for designating a background color of an AV scene, a property 
variable for designating a window of a scene in which the AV scene is 
synthesized with a markup document scene, a property variable for 
designating a trimming area of the scene in which the AV scene is 
synthesized with the markup document scene, and a property variable 
for designating an area to increase and reduce a desired portion of the 
AV scene are defined in an object for controlling the above-mentioned 
CSS file. 

In addition, the AV scene may be increased/reduced according to 
a user's input using an object source code of an application program 
interface (API) for a document object model (DOM). The value of the 
object source code needed in performing this operation can be referred 
to using a script language in the markup document. 

The following object source code is used to bind 
"ScreenDisplayProperties" in root elements (i.e., <frameset> and <html>) 
of the uppermost level. 



Interface ScreenDisplayProperties { 

attribute ScreenDisplayRule screenDisplaylnfo; 



. Here, "ScreenDisplayProperties" are connected to root elements 
of a markup document, and the value of "ScreenDisplayProperties" -can 
be referred to using a script language in the markup document. An 
example of the object source code is as follows. 



IDL Definition 

Interface ScreenDisplayRule 
{ 

attribute unsigned short videoPlacement; 
attribute DOMString colorBackground; 
attribute DOMString viewport; 
attribute DOMString window; 
attribute DomSting videoViewport; 

li 

Attributes 

videoPlacement: It designates a display mode of a DVD-video. That is it represents that 

const unsigned short VIDEO_PLACEMENT_NONE = 0; 
const unsigned short VIDEO PLACEMENT EMBEDDED = 1- 
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if (evt == 2) 






{ // evt == 2 : Reduce an AV scene to a designated size 
vdi = documentdocumentElementscreenDisplaylnfo; 
vdi.videoPlacement = 1 ; 
vdi.colorbackground = "black"; 
vdi.videoviewport = M rect(0px,71 9px,479px,0px)" 
vdi.window="rect(1 Oyo.QOyo.QO^o,! 0%) 

// Here, a % value is a relative value for "width" and "height" defined by the 

tag "Object". 
} 

} 

— > 

</script> 
</head> 

<body onload="dvdvideo.playO; M> 
<table border="0" width="72CT height="480"> 
<tr> 

<td width="277" height="184" align="left" valign="top"> 

<objectdata="dvd": width="277" height="184" border ="07> 
</td> 

<td width="443" height="480" align="left" valign="top" rowspan="2"> 
<img src="b1.png" width="443" height="480" border="07> 

</td> 
</tr> 
<tr> 

<td width="277" height="296"> 

<inputtype="button" value="2oom-ln" onClick="zoom(0)7> 
<input type="button" value="Restore" onClick="zoom(1)7> 
<input type="button" value="Zoom-Out" onClick="zoom(2)7> 
</td> 
</tr> 
</table> 
</body> 

</html> 





In this way, the scale of the AV scene can be manipulated through 



a user's input using the API for a DOM and the script language. 

Accordingly, the AV scene and the markup document scene are 
freely synthesized with each other according to the above-mentioned 
embodiment, and the display location of the AV scene can be set by a 
style sheet linked to the markup document or embedded in the markup 
document, thereby allowing a user to enjoy scenes in various shapes. 

A reproducing method according to the present invention will be 
described below based on the above structure. 

FIG. 9 shows a flowchart illustrating a preferred embodiment of a 
reproducing method according to the present invention. Referring to 

21 
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sheet-ftle^e-AV^cene-is-oy-eriappeil- on the markup document scene — 

and is displayed. In step 905-2, when the embedded mode is described 
in the style sheet file, the AV scene is embedded in the markup 
document scene and is displayed. In step 905-3, when the background 
5 mode is described in the style sheet file, the markup document scene is 
overlapped on the AV scene and is displayed. 

FIG. 10 shows a flowchart specifically illustrating steps 
subsequent to step 903 of FIG. 9. Referring to FIG. 10, the 
presentation engine 3 interprets the CSS file in the markup document or 
1 0 the link tag and calls the CSS file in step 1 001 , interprets the called CSS 
file in step 1002, and reads the display mode designation information 
and the AV trimming area designation information, which are described 
in the CSS file in step 1003. The blender 4 blends and outputs the 
command and the markup document scene provided from the 
1 5 presentation engine 3 and the AV scene provided from the AV decoder 2, 
and the markup document scene and the AV scene are displayed 
according to the display mode described in the CSS file in step 1004. 
In the case of a none mode, the AV scene is not displayed in step 
1 004-1 . Otherwise, one of the PIP mode (step 1 004-2), the background 
20 mode (step 1004-3), and the embedded mode (step 1004-4) may be 
displayed. 

FIG. 11 shows a flowchart illustrating a preferred embodiment of 
a recording method according to the present invention. Referring to 
FIG. 11, the recording apparatus records the AV data on an information 
25 storage medium (step 1101), records the markup document that is to be 
reproduced with the AV data on the information storage medium (step 

1102) , and records the above-mentioned scene synthesis information in 
the style sheet file linked to or embedded in the markup document (step 

1103) . In step 1103, an object having property variables for controlling 
30 the style sheet file in the markup document and a program for controlling 
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screen aspect retire so tn correspon d to a dis pla y having different 

aspect ratios. 

In order to display interactive contents that are not distorted in the 
display having different aspect ratios as shown by CD and ®, the 

5 manufacturer prepares respective markup documents by considering 
different aspect ratios of the display as shown by © and <D, and 
respective AV data (i.e., AV contents of DVD-video format) by 
considering different aspect ratios of the display as shown by © and ©. 
Contents of © + © in which AV data for 4 x 3 is synthesized with a 

10 markup document for 4 x 3 can be displayed on the screen of a display 
for 4 x 3 as shown by ®, and contents of ® + © in which AV data for 
16 x 9 is synthesized with a markup document for 16 x 9 can be 
displayed on the scene of a display for 16 x 9 as shown by <2), as 
contents which the manufacturer wants, without performing any special 

15 operation. 

Second displaying method: The manufacturer prepares respective 
markup documents by considering aspect ratios of a display, and AV 
data corresponds to a display having different aspect ratios in the 
following way when contents having a fixed aspect ratio are 
20 manufactured. 

When only ©, ©, and © are manufactured 

- When displaying on a display for 4x3: The contents of 

© + © in which the AV data for 4 x 3 is synthesized with the markup 
document for 4 x 3 can be displayed in a scene ®, as contents which 
25 the manufacturer wants, without performing any special operation. 

- When displaying on a display for 1 6 x 9: The AV data 

© for 4 x 3 is embedded in the markup document for 16 x 9 including 
right and left blanks without expansion. Only the right and left blanks 
may be eliminated by setting the size of an object tag of the markup 
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riiffemnra. in picture beauty of the 4x3 pa n & scan scene de pending on 

an editor's skill. 

Letter box is a picture manufactured by inserting a black band in 
upper and lower portions of a scene and reducing the scene so that the 
5 user can effectively view a picture manufactured in a wide scene format 
(16 x 9) in a normal display (screen ratio 4 x 3). The upper and lower 
black bands of the picture are referred to as "Matte". 

- When displaying on a display for 16 x 9: The contents of 

<D + © in which AV data for 16 x 9 is synthesized with a markup 
10 document for 16 x 9, can be displayed in a scene 2, as contents which 
the manufacturer wants, without performing any special operation. 

Third displaying method: The manufacturer prepares respective 
AV data by considering aspect ratios of a display, and a markup 
document corresponds to a display having different aspect ratios in the 
15 following way when contents having a fixed aspect ratio are 
manufactured. 

When only ©, ©, and © are manufactured 

- When displaying on a display for 4x3: The contents of 

© + © in which AV data for 4 x 3 is synthesized with a markup 
20 document for 4x3, can be displayed in a scene 1 , as contents which the 
manufacturer wants, without performing any special operation. 

- When displaying on a display devcie for 16 x 9: AV data 6 for 
16 x 9 is converted into a shape of a 4 x 3 normal scene, 4x3 letter box 
scene, or 4 x 3 pan & scan scene and then is embedded in a markup 

25 document 3 for 4 x 3. The manufactured contents of 3 + 6 can be 
displayed to include blanks on the right and left sides of a scene 2 
through middle alignment. 

When only ©, ©, and © are manufactured 

- When displaying on a display for 4 x 3: AV data 5 for 4 x 
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display for 4 x 3 through a concept of "hot area" and an API with respect 

to aspect ratio conversion. In this case, preferably, the pan & scan or 
letter box is applied to the video mode other than the interactive mode. 
- When displaying on a display for 16 x 9: The contents of 

5 ® + © in which AV data is synthesized with a markup document, can 
be displayed in a scene @ as the manufacturer wants, without 
performing any special operation. 

Here, "hot area" and the API with respect to the aspect ratio 
conversion will be described in greater detail. 

10 When interactive contents manufactured for use in 16 x 9 

(resolution of 854 x 480) are displayed on a display for 4 x 3 (resolution 
of 640 x 480), only a 640 x 480 area of the entire area of the interactive 
contents is displayed on the display for 4x3, the other 214 x 480 area is 
not displayed on the display for 4x3. Therefore, the manufacturer 

15 includes contents which should be displayed on a 640 x 480 area (in the 
present invention, the displayed maximum area is referred to as "hot 
area" regardless of an aspect ratio of a display), which can be displayed 
regardless of an aspect ratio of a display during initial layout, and 
includes unimportant contents or does not include any contents in the 

20 other area. In order to display a markup document manufactured by the 
concept of "hot area" on a display for 4 x 3 and in order to display an AV 
scene and a markup document scene in various ways, interactive 
contents are displayed using the above-mentioned properties "viewport:" 
and "window:" used in the object source code of the API for a DOM, and 

25 descriptions thereof will be described later. 

Hereinafter, a simple example in which interactive contents for 16 
x 9 are displayed on a display for 4 x 3 regardless of an aspect ratio of a 
display, will be described with reference to FIG. 14. 

The manufacturer constitutes a markup document using the 

30 concept of "hot area" so as to display a scene that is not distorted 
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The above Equations 1 through 4 are based on a marku p 

document, but if a lower subscript "document" is changed into "video", 
the definition of AV data is made. 

An example of an optimum table showing the size of each area 

5 through the above-mentioned displaying methods at a pixel size ratio of 
1 x 1 to easily recognize a document area, a viewport area, a window 
area, and a screen area and the size of an area through other available 
displaying methods is as follows. 



Case 


Document 
area 


Viewport 
Area 


Window 
area 


Screen 
Area 


When interactive contents 
manufactured for 640 x 480 are 
displayed on a 640 x 480 display 


640 x 480 


640 x 480 


640x480 


640 x 480 


When interactive contents 
manufactured for 640 x 480 are 
displayed on a 854 x 480 display 


640x480 


640 x 480 


640 x 480 


854x480 


When interactive contents 
manufactured for 640 x 480 are 
displayed on a 854 x 480 display 


640 x 480 


640 x 480 


854 x 480 


854x480 


When interactive contents 
manufactured for 854 x 480 are 
displayed on a 640 x 480 display 


854 x 480 


854 x 480 


640x480 


640 x 480 


When interactive contents 
manufactured for 854 x 480 are 
displayed on a 640 x 480 display 


854x480 


854 x 480 


640x480 


640x480 


When interactive contents 
manufactured for 854 x 480 are 
displayed on a 640 x 480 display 


854 x 480 


640 x 480 


640x480 


640x480 


When interactive contents 
manufactured for 854 x 480 are 
displayed on a 640 x 480 display 


854X480 


854x480 


854 x 480 


854x480 



10 Meanwhile, reproducing methods according to an aspect ratio are 

largely classified by a static method using a CSS and a dynamic method 
using an API for a DOM. When interactive contents are initially 
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tbe-AV— scene— js-syntl^sized_wjth_tl3e^arkup^ 

displayed. A value is <shape>, and an initial value is rect (0%, 100%, 
100%, 0%). 

6. viewport: It designates a trimming area of the scene in which 

5 the AV scene is synthesized with the markup document scene. A value 

is <shape>, and an initial value is rect (0%, 100%, 100%, 0%). 

@screen-display 4x3N 
{ 

background-color : #000000 
viewport : (0px,629px,479px,90px) 
window : (0px,719px,479px,0px) 
video-viewport : (0px,719px,479px,0px) 

} 

@screen-display 4x3L 
{ 

background-color : #000000 
viewport : (0px,71 9px,479px,0px) 
window : (60px,719px,419px,0px) 
video-viewport : (0px,719px,479px,0px) 

> 

@screen-display 4x3P 
{ 

background-color : #000000 
viewport : (0px,629px,479px,90px) 
window : (0px,71 9px,479px,0px) 
video-viewport : (0px,629px,479px,90px) 

} 

@screen-display 16x9W 

background-color : #000000 
viewport : (0px,719px,479px,0px) 
window : (0px,719px,479px,0px) 
video-viewport : (0px,719px,479px,0px) 

} 



Since the default style sheet is differently set in each reproducing 
apparatus, a scene cannot be usually displayed as the manufacturer 
1 0 wants. Thus, preferably, the manufacturer makes an additional CSS in 
the markup document and attaches the CSS to the document so that the 
user can effectively display the AV data (DVD-video) and the markup 
document manufactured at a fixed aspect ratio, even at an aspect ratio 
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1 






</style> 
</head> 

<body id="bodyNode"> 

</body> 
</html> 




5 


In this way, documents initially displayed by statically applying the 
CSS are displayed in a scene through the structure of processing shown 
in the following table in the presentation engine, and a corresponding 
page of a document of which aspect ratio is changed during reproduction 
should be reloaded to guarantee a right scene. If the corresponding 
page of the document is not reloaded and is displayed in the scene, the 
scene of the markup document may be distorted, or the markup 
document may be not mapped to the AV data. 




If (Initial Display aspect ratio— 4x3 && (no video == 1 current display == noma!)) 
{ 

apply@screen-display 4x3N 
else if (Initial Display aspect ratio==4x3 && current display — 4x3P) 
{ 

apply@screen-display 4x3P 
else if (Initial Display aspect ratio==4x3 && current display ™ 4x3L) 
{ 

apply@screen-display 4x3L 
else if (Initial Display aspect ratio==16x9) 
{ 

apply@screen-display 16x9W 

} 





10 



In the above algorithm in the presentation engine, "Initial Display 
aspect ratio" corresponds to Initial Display aspect ratio of a system 
parameter SPRM(14) defined in DVD Specifications for Read-Only 
Disc/Part 3. "Initial Display aspect ratio" is a parameter obtained by 
15 initially setting an aspect ratio which the user prefers and has the values 
of two types of 4 x 3 and 16 x 9, as shown in the above algorithm. 

A "no_video" element indicates whether there is AV data (i.e., 
DVD-video) in a markup document loaded initially. If the value of 
"no_video" element is "0", it is determined that DVD-video is included in 

35 



BNSDOCID: <WO 0303646QA1J_> 



WO 03/036460 



PCT/KR02/01977 











window: It has the value of <shape> as a window area on a display to which the trimmed 
markup document is to be mapped. 

videoviewport: It has the value of <shape> as a trimming area of AV data. 





The above-mentioned dynamic definition using the object source 
of the API for the DOM is implemented by a script language included in 
the markup document, as shown in the following example. The 
5 example is made by the manufacturer by considering event handling 
according to user's aspect ratio conversion of AV data for 16 x 9 (i.e., 
DVD-video) and a markup document for 16 x 9, which are to be 
displayed in an embedded mode. 

<?xml version "1.0" encoding="UTF-8"?> ' ~ ~~ 

<!DOCTYPE html PUBLIC "-//DVD//DTD XHTML DVD-HTML 1 .0//EN" 
http://www.dvdforum.org/envideo/dtd/dvdhtml-1-0.dtd"> 

<html> 
<head> 

<title>Example of aspect ratio change</title> 

<script type="text/javascript"> 

<— 

function eventHandler(evt) 
{ 

var vdi; 

if (evtindex == SCREEN_DISPLAY__MODE_CHANGE && (evtparaml == 0)) 
{//paraml ==0 :4x3N 

vdi = documentdocumentElement.screenDisplaylnfo; 

vdi.videoPlacement = 1 ; 

vdi.colorBackground = "black"; 

vdi.viewport = M (0px,629px f 479px,90px)"; //An area formed by cutting right 
and left sides of a markup document is selected. In this case, it is most preferable that 
the selected area is consistent with a "hot area". 

vdi.window = "(0px,71 9px,479px,0px) M ; 

vdi.videoviewport = "(0px,71 9px,479px,0px) M 

} 

if (evtindex == SCREEN JDISPLAYJVIODEJ2HANGE && (evtparaml == 1)) 
{//paraml == 1 :4x3L 

vdi = documentdocumentElementscreenDisplaylnfo; 

vdi.videoPlacement = 1 ; 

vdi.colorBackground = "black"; 

vdi.viewport = "(0px,629px,479px,90px)"; 

vdi.window = "(0px t 719px > 479px,0px)"; 

vdi.videoviewport == "(0px,719px,479px,Opx) M // Even though the entire 
DVD-video picture is selected, in a 4 x 3 letter box mode, it seems that "Matte" is added to 
the upper and lower portions of a scene. 

} 
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I^G^1fi,-by-using^e^bov^^ 

the markup document scene is displayed according to a screen mode 
(aspect ratio, resolution, and video output method) set by the user or set 
in the reproducing apparatus. The screen mode can be changed even 

5 during reproduction through a user's input. A viewport value and a 
window value of the markup document applied in this case can be 
applied to a next markup document as it is if these are not changed in a 
next markup document. 

In step 1601, the presentation engine 3 reads the screen mode 

10 (aspect ration, resolution, and video output method) set in the 
reproducing apparatus or the screen mode set by the user. Here, a 
video output method means that a 16 x 9 video is output in a 4 x 3 letter 
box or 4 x 3 pan & scan, and 16x9 wide mode, which can be displayed 
without scene distortion. 

15 When a display is set to a 4 x 3 size, the AV decoder 2 converts 

an AV stream into a letter box or pan & scan shape and outputs the 
converted AV stream. When the display is set to a 16 x 9 size, the AV 
decoder 2 outputs the AV stream in a 16 x 9 size without conversion. 
This is because the AV stream is generally encoded in the 16x9 size in 

20 the DVD reproducing apparatus. However, it is preferable that the 
output is performed only when the AV stream is reproduced in a video 
mode other than a conventional interactive mode, and in the interactive 
mode, the AV stream is always output in a 16 x 9 size. 

In step 1602, a default style sheet in the presentation engine 3 is 

25 selected based on the set screen mode, and properties such as viewport, 
window, and video viewport, which are defined in the corresponding 
default style sheet, are determined. 

The, presentation engine 3 interprets the markup document read 
by the reading unit 1 and checks a style sheet linked to or embedded in 

30 the markup document. In step 1603, if there is no style sheet provided 
by the manufacturer in the markup document, the presentation engine 3 
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without scene distortion, the user can enjoy a display state the closest to 
manufacturer's intention, and simultaneously, using the markup 
document having a fixed aspect ratio (screen ratio), a disc space can be 
5 more effectively used by avoiding repeated recording of interactive 
contents. 

First, when a storage medium, in which AV data (DVD-video) is 
manufactured to a 16 x 9 size and a markup document is manufactured 
to a 4 x 3 size, is displayed on a display for 16 x 9 through a reproducing 

10 apparatus, displaying methods according to displaying modes 
(embedded mode, background mode, and PIP mode) will be described 
with reference to FIG. 17. 

First, a case where AV data for 16 x 9, as shown in (a) of FIG. 17, 
is synthesized with a markup document in an "embedded mode" 

15 embedded by "object" elements, as shown in (b) of FIG. 17, will be 
described. If the manufacturer sets "width" and "height" of the "object" 
tag in the markup document so that the AV data for 16 x 9 is expressed 
as it is, the entire synthesized scene is selected as a viewport area and 
is not expanded so that there is no scene distortion in a display for 16 x 

20 9, and an area excluding right and left blanks is selected as a window 
area, and the synthesized scene is displayed, thereby the scene can be 
o effectively displayed. However, if the manufacturer sets "width" and 
"height" of the "object" tag used to embed the AV data for 16 x 9 to a 4 x 
3 screen ratio, the AV data will be embedded in the markup document in 

25 a normal shape, letter box or pan & scan shape, and the synthesized 
scene will be displayed on the display for 16 x 9 without expansion. In 
this case, even though the AV data is manufactured to a 16x9 size, the 
AV data is embedded in the display for 16 x 9 at the 4 x 3 screen ratio, 
and thus this case is not considered as preferable. 

30 Second, a case where the AV data is synthesized with the markup 

document in a "background" mode using the property video-placement: 
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dispJayed-on-the-display-4iaving-the-1-6-X-9-aspect-ratio= — Since-only-tbe 

window area of the synthesized scene is defined when displaying the 
scene even though the manufacturer encodes the AV data to a 16x9 
size, in order to prevent the AV data from displaying in the letter box or 
5 pan & scan shape, as shown in (a) and (b) of FIG. 18, in another 
embodiment of the present invention, a scene can be displayed in the 
most appropriate shape according to an aspect ratio of a display or 
display modes (embedded mode, background mode, and PIP mode) by 
defining viewport and window properties only for a markup document 

1 0 and viewport and window properties only for AV data, respectively. 

In FIG. 19, assuming that markup documents considering different 
aspect ratios of displays (3) and © and AV data (i.e., AV contents of 
DVD-video format) considering different aspect ratios of displays © and 
© exist in displays © and (2) for 4-x 3 and 16 x 9, resolution of 

15 displays is 720 x 480 having the same pixel number, as shown in FIG. 

20, in the case of NTSC. Only, in a 16 x 9 aspect ratio, the ratio of 
width to length of a pixel is 1 .78:1 , as shown in (a) of FIG. 20, and in a 4 
x 3 aspect ratio, the ratio of width to length of a pixel is 1 .33:1 . 

In this way, if a display is a television (TV), the aspect ratio of a 
20 scene pixel varies according to the type of TV, and thus it is difficult to 
understand variations in a scene. Thus, for the convenience of 
explanation, a conversion equation with regard to each case of 1.78 x 1, 
1.33 x 1, and 1 x 1 screen ratios will be described with reference to FIG. 

21 . Only, the length of a unit pixel is the same. Namely, in all cases, 
25 there are no variations in height. 

Conversion CD: The resolution of 720 x 480 having a 16 x 9 pixel 
aspect ratio is converted into a shape having a 1 x 1 unit pixel aspect 
ratio. 

Total horizontal resolution 
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= normalization (when the length ratio of the unit pixel is 1 , a width 
ratio, total vertical resolution) 
= when the length ratio of the unit pixel is 1 , a width ratio x total 
5 horizontal resolution 

= 1 .33 x 480 
= around 640 

Therefore, the resolution of 720 x 480 having a 4 x 3 pixel aspect 
ratio is consistent with the resolution of 640 x 480 having a 1 x 1 unit 
10 pixel aspect ratio. That is, when converting into a (E> shape, the 
following Equation is formed: 

X(1.33:l) = — X(l:l) 
640 v J 

640 

X(l:l) = — JT(1.33:1) 
720 ~ ' 

Conversion ©: Conversion between the resolution of 720 x 480 
having a 16 x 9 pixel aspect ratio and the resolution of 720 x 480 having 
15 a 4 x 3 pixel aspect ratio 

The following Equation is formed between the horizontal 
resolution having the 4 x 3 (12 x 9) pixel aspect ratio and the horizontal 
resolution having the 16 x 9 pixel aspect ratio. 

X(1.7S:V)=— X(1.33:l) 
16 

X(1.33:l) = i|x(1.78:l) 

20 That is, it is most appropriate that the 720 x 480 resolution in the 

display for 4 x 3 is displayed in 540 x 480 resolution in the display for 16 
x 9, and thus when only an are (corresponding to "hot area") 
corresponding to 540 x 480 in the interactive contents for 16 x 9 is 
selected, the selected area is rightly displayed on the display for 4x3. 

25 Meanwhile, if the 720 x 480 resolution having the 16x9 pixel 

aspect ratio is converted into a letter box shape in the 720 x 480 
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in which distortion in which a scene seems to be slim does not occur, 
unlike a conventional normal shape. 

4 x 3L: If a user sets a screen output to a 4 x 3 letter box shape 
5 4 x 3P: If a user sets a screen output to a 4 x 3 pan & scan shape 

16 x 9W: If a user sets a screen output to a 16 x 9 wide shape 

2. "video-placement" property 

It designates a display mode of the AV scene. None, embedded, 
pip-#, and background represent nothing displayed, embedded mode, 
10 PIP mode, and background mode, respectively. An initial value is an 
embedded mode. 

3. background-color property 

It designates a background color of a scene formed of a single 
color. A value is <color>, and an initial value may vary according to a 
15 user agent (U A). 

4. "document-viewport" property 

It designates a trimming area of a markup document. A value is 
<shape>, and an initial value is rect (0%, 100%, 100%, 0%). Here, the 
value of the defined <shape> is rect(<top> I <right>,<bottom>,<left>). 
20 5. "document-window" property 

It designates an area of a window in which a markup document is 
displayed on a scene of a display. A value is <shape>, and an initial 
value is rect (0%, 100%, 100%, 0%). 

6. "video-viewport" property 

25 It designates a trimming area of the scene in which the AV scene 

is synthesized with the markup document scene. A value is <shape>, 
and an initial value is rect (0%, 100%, 100%, 0%). 

7. "video-window" property 

It designates an area of a window in which AV data is displayed 
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©screen-display 16x9W 
{ 

background-color : #000000 
document-viewport : (0px,719px,479px,0px) 
document-window : (0px,719px,479px,0px) 
video-viewport : (0px,719px,479px,0px) 
video-window : (0px,719px l 479px,0px) 

} 



Since the default style sheet is differently set in each reproducing 



apparatus, a scene cannot be usually displayed as the manufacturer 
wants. Thus, preferably, the manufacturer makes an additional CSS in 
the markup document and attaches the CSS to the document so that the 
5 AV data and the markup document can be effectively displayed even at 
an aspect ratio set by the user. The following example shows how the 
manufacturer makes the CSS in the markup document so that AV data 
for 16 x 9 and a markup document for 4 x 3 can be effectively displayed 
in a "background mode". The CSS may be made using a "style" tag, as 
10 shown the following example, and may be used through external 
reference using a "link" tag. 

<!DOCTYPE html PUBLIC "-//DVD//DTD XHTML DVD-HTML 1 .0//EN" 

"http://www.dvdforum.org/enav/dtd/dvdhtml-1 -0.dtd"> 

<html> 
<head> 

— <title>Example of aspect ratio change</title> 
<style type="text/css"> 

©screen-display 4x3N 
{ 

video-placement : background 
background-color : #000000 
document-viewport : (0px,719px,479px,0px) 
document-window : (0px,719px,479px,0px) 

video-viewport : (0px,629px,479px f 90px) // If the entire area is selected using a 
viewport area, AV data displayed as a background seems to be slim. Thus, in order to 
solve this problem, the manufacturer just selects pan & scan. A selection area may be 
varied by the manufacturer. 

video-window : (0px,719px,479px,0px) 

} 

©screen-display 4x3L 
{ 

video-placement : background 
background-color : #000000 
document-viewport : (0px,719px,479px,0px) 
document-window : (60px.719px,419px,0px) 
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Herer^SGreenDisplayProperties— are-connected-to-root-elements- 

of a markup document, and the value of "ScreenDisplayProperties" can 
be referred to using a script language in the markup document. 
The definition and property of an interface definition language 

5 (IDL) are as follows. 



IDL Definition 

Interface ScreenDisplayRuIe 
{ 

readonly attribute unsigned short screenDispIayMode; 
attribute unsigned short videoPlacement; 
attribute DOMString colorBackground; : 

attribute DOMString documentviewport; 
attribute DOMString documentwindow; 
attribute DOMString videoviewport; 
attribute DOMString videowindow; 

}; 

Attributes 

screenDispIayMode: An aspect ratio of an output scene set by a user 

const unsigned short SCREEN J3ISPLAYJVIODE_4X3NORMAL = 0; 
const unsigned short SCREEN_DISPLAY_MODE_4X3LETTERBOX = 1 ; 
const unsigned short SCREEN JDISPU\Y__MODE_4X3PAN&SCAN = 2; 
const unsigned short SCREEN JD!SPLAYJVIODE_16X9WIDE = 3; 

videoPlacement: It designates a display mode of a DVD-video 
const unsigned short VIDEO_PLACEMENT_NONE = 0; 
const unsigned short VIDEO_PLACEMENT_EMBEDDED = 1 ; 
const unsigned short VIDEO_PLACEMENTJBACKGROUND - 2; 
const unsigned short VIDEO_PLACEMENT_PIP = 3; 

colorBackground: It has the value of <color> as a background color of a DVD-video 
scene. 

documentviewport: It has the value of <shape> as a trimming area of a markup 
document. 



documentwindow: It has the value of <shape> as a window area on a display to 
which the trimmed markup document is to be mapped. 

videoviewport: It has the value of <shape> as a trimming area of the DVD-video. 

videowindow: It has the value of <shape> as a window area on a display to which 
the trimmed DVD-video is to be mapped. However, when videoPlacement is in an 
embedded mode, the window area is restricted by "width" and "height" defined by an 
<object> tag in the markup document. 
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vdi.colorBackground = "black"; * 






vdi.documentviewport = "(Opx^Qpx^TQpx.QOpx)"; 
vdi.documentwindow = H (0px,719px,479px,0px) M ; 

vdi.videoviewport = "(Opx^px^px^Opx)" // Part of a scene formed by 
cutting right and left sides of the DVD-video picture will be selected. 
} 

if (evt.index == SCREEN DISPLAY MODE CHANGE && (evtparaml ==3)) 
{//paraml ==3: 16x9W 

vdi = documentdocumentElementScreenDisplaylnfo; 

vdi.videoPlacement = 1 ; 

vdi.colorBackground = "black"; 

vdi.documentviewport = "(0px,719px l 479px,0px)"; 

vdi.documentwindow = "(0px,71 9px,479px,0px)"; 

vdi.videoviewport = M (0px,71 9px,479px,0px)" 

} 
} 

</script> 

<script type="text/javascript"> 
<— 

function setupEventHandlerO 
{ 

// eventHandler is registered to bodyNode and Interactive Contents 
// SCREEN_DISPLAY__MODE_CHANGE == 500 
bodyNode.addEventListener("dvdvideo" l eventHandler > true); 

u vu w iucu.ouuol.1 iL/c I UI-VpI li\OUr\CCIM UlOr Ln T IvlKJLJtl wrir\l>iV3t,irUey ,/ 

</script> 
</head> 

<body id="bodyNode M onload="setupEventHandlerO"> 

</body> 
</html> 





The markup document is displayed in a scene through initial static 
definition, and then due to the occurrence of an event according to a 
user's aspect ratio conversion key (or button) input, 
5 "vdLscreenDisplayMode" information is read, thereby reconstituting the 
aspect ratio of the scene using a script language included in the above 
markup document. 

FIG. 23 shows a flowchart of another preferred embodiment of a 
reproducing method according to the present invention. Referring to 
10 FIG. 23, by using the above-mentioned static and dynamic definitions, 
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scneen-mode t _andjfJhere_is„a_sty.le-sheet_pn 

in the markup document, the presentation engine 3 outputs the markup 
document to the scene, using the properties such as document viewport, 
document window, video viewport, and video window, according to 
5 @screen-display, which are defined in the corresponding style sheet. 

In step 2304, it is determined whether the screen mode according 
to a user's aspect ratio conversion key (or button) is changed. In step 
2305, if the screen mode is changed by the user, the presentation 
engine 3 informs the corresponding markup document of an aspect ratio 

10 conversion event ASPECTJRATIO_CHANGE, executes a script caused 
by the event, interprets a screen display property variable corresponding 
to the changed screen mode using ScreenDisplayProperties in the 
presentation engine 3, changes a screen output state of the markup 
document based on the interpreted information, and outputs a new 

15 markup document to the scene. In step 2306, if the screen mode is not 
changed in step 2304, it is determined whether the output of the markup 
document is terminated, and the output of the markup document is 
terminated. 

The above recording and reproducing methods can be 
20 implemented with a computer program. Program codes and code 
segments of. the computer program can be easily made by a computer 
programmer skilled in the art. Also, the above program is stored in 
information storage media (computer readable media), read and 
executed by the computer, thereby performing a method for recording 
25 and reproducing a markup document and AV data. The information 
storage media include magnetic recording media, optical recording 
media, and carrier waves. 

Industrial Applicability 
30 As described above, according to the present invention, an 

information storage medium including AV data and a markup document 
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informat ion includes a li nk tag reco rded in the markup document and a 

cascading style sheet (CSS) inserted in the link tag. 

8. The medium of claim 7, wherein the CSS includes display 
5 mode designation information for designating a display mode of the AV 

scene. 

9. The medium of claim 8, wherein the CSS further includes an 
AV trimming area designation information for designating an area to 

10 increase and reduce a desired portion of the AV scene. 

10. The medium of claim 8, wherein the CSS further includes 
background color designation information for designating a background 
color of the AV scene. 

11 . The medium of claim 8, wherein the CSS further includes 
trimming area designation information for designating a trimming area of 
a scene in which the AV scene is synthesized with the markup 
document. 

12. The medium of claim 8, wherein the CSS further includes 
window designation information for designating a window in which a 
scene where the AV scene is synthesized with the markup document is 
displayed on the screen of a display. 

1 3. The medium of claim 8, wherein the CSS further includes 
screen display type information representing an aspect ratio of a scene 
in which the AV scene is synthesized with the markup document scene. 

30 14. The medium of claim 8, wherein the CSS further includes 

window designation information of the AV scene for designating an area 
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further inclu de a screen display mode representin g an aspect ratio of a 
scene in which the AV scene is synthesized with the markup document 
scene. 

22. The medium of claim 16, wherein the property variables 
further include a property variable for designating a window in which a 
scene where the AV scene is synthesized with the markup document is 
displayed on the screen of the display. 

23. The medium of claim 1 , further comprising reproduction 
control information for the AV data, and the AV data is decoded as the 
AV scene by referring to the reproduction control information. 

24. The medium of claim 1 , wherein the AV data and the 
reproduction control information are recorded in a video directory, and 
the markup document and the scene synthesis information are recorded 
in an interactive directory. 

25. An information storage medium comprising: 
AV data including audio data and video data; 

a markup document; and 

scene synthesis information in which a markup document scene 
obtained from the markup document is synthesized with an AV scene 
obtained from the AV data without scene distortion corresponding the 
change of a screen mode. 

26. The medium of claim 25, wherein the screen synthesis 
information is stored in a cascading style sheet (CSS) file linked to the 
markup document using a link tag or embedded in the markup document 
using a style tag. 
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► and unimportant contents are recorded or any contents are not recorded 

in other area. 

31 . The medium of claim 30, wherein when the markup 

5 document having a first aspect ratio is displayed on a display having a 
second aspect ratio having a resolution lower than that of the first aspect 
ratio, the hot area is selected using the trimming area designation 
information of the synthesized scene, and the selected hot area is 
mapped to a designated area on the screen of the display using the 
1 0 window designation information of the synthesized scene. 

32. The medium of claim 26, wherein the screen synthesis 
information includes at least one of screen display type information for 
designating an aspect ratio of a scene in which the AV scene is 

15 synthesized with the markup document scene, display mode designation 
information for designating a display mode of the AV scene, background 
color designation information for designating a background color of the 
scene, markup document trimming area designation information for 
designating a trimming area of the markup document, markup document 

20 window designation information for designating a window in which the 
markup document scene is displayed on a screen of a display, AV 
trimming area designation information for designating an area to 
increase and reduce a desired portion of the AV scene, and window area 
designation information of the AV scene for designating a window in 

25 which only the AV scene is displayed on the screen of the display. 

33. The medium of claim 32, wherein when the AV data having 

a first aspect ratio is displayed on a display having a second aspect ratio 
having a resolution lower than that of the first aspect ratio, the scene 
30 synthesis information is set so that the AV data having the first aspect 
ratio is output without conversion. 
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designating_aJrimming_areaJo^ a desired portion of 

the AV scene, a property for designating a window in which the AV 
scene is displayed on the screen of the display. 

5 38. The medium of claim 37, wherein in response to user's 

screen mode conversion, the program using an object source controls a 
property for designating a trimming area of the synthesized scene, a 
property for designating a window area of the markup document scene, a 
property for designating a trimming area of the AV scene, and a property 
10 for designating a window area of the AV scene according to the screen 
display mode, thereby reconstituting the aspect ratio of the scene. 

39. The medium of claim 25, wherein conversion between a 
screen ratio and an aspect ratio according to the screen mode includes 

15 conversion of a 720 x 480 resolution having a 16 x 9 pixel aspect ratio 
into a 1 x 1 unit pixel aspect ratio, conversion between a 854 x 480 
resolution and 640 x 480 resolution each having a 1 x 1 unit pixel aspect 
ratio, conversion of a 720 x 480 resolution having a 4 x 3 pixel aspect 
ratio into a 1 x 1 unit pixel aspect ratio, conversion of a 720 x 480 

20 resolution having a 4 x 3 pixel aspect ratio into a 1 x 1 unit pixel aspect 
ratio, and conversion between a 720 x 480 resolution having a. 16 x 9 
pixel aspect ratio and a 720 x 480 resolution having a 4 x 3 pixel aspect 
ratio. 

25 40. The medium of claim 27, wherein the screen display type 

information includes 4x3 normal, 4x3 letterbox, 4x3 pan & scan, and 
16x9 wide. 

41 . A method for recording AV data including audio data and 
30 video data on an information storage medium, the method comprising: 
(a) recording the AV data; 
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> 46. The method of claim 45 , wherein the dis pla y mode 

designation information indicates .at least one of an embedded mode in 
which at least the AV scene is embedded in at least part of the markup 
document scene, a picture in picture (PIP) mode in which the AV scene 
5 is overlapped on the markup document scene, and a background mode 
in which the markup document scene is overlapped on the AV scene. 

47. The method of claim 45, wherein in step (b), contents which 
should be displayed, are recorded in a maximum area shown regardless 

10 of an aspect ratio of the display, that is, a hot area, and unimportant 
contents are recorded or any contents are not recorded in the other area. 

48. The method of claim 47, wherein when the markup 
document having a first aspect ratio is displayed on a display having a 

15 second aspect ratio having a resolution lower than that of the first aspect 
ratio, the hot area is selected using the trimming area designation 
information of the synthesized scene, and the selected hot area is 
mapped to a designated area on the screen of the display using the 
window designation information of the synthesized scene. 

20 

49. The method of claim 45, wherein the scene synthesis 
information further includes window designation information of the AV 
scene for designating a window in which the AV scene is displayed on 
the scene of the display. 

25 

50. The method of claim 43, wherein in step (c), an object 
having property variables for controlling the CSS and a program for 
controlling the CSS on the basis of the object are recorded. 

51 . The method of claim 50, wherein the property variables 

30 include at least one of a screen display mode for designating an aspect 
ratio of a scene in which the AV scene is synthesized with the markup 
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a propert y for desi gnatin g a window area of the markup document scene . 

a property for designating a trimming area of the AV scene, and a 
property for designating a window of the AV scene area according to the 
screen display mode, thereby reconstituting the aspect ratio of the 
5 scene. 



55. A method for reproducing AV data including audio data and 
video data recorded on an information storage medium, the method 
comprising: 

10 (a) interpreting a markup document to be displayed together 

with the AV data; 

(b) interpreting scene synthesis information which describes 

at least two display modes for displaying a markup document scene 
obtained by reproducing the markup document and an AV scene 
15 obtained by reproducing the AV data together; and 

(c) displaying the AV scene and the markup document scene in 
one of the display modes according to the interpreted scene synthesis 
information. 



56. The method of claim 55, wherein step (c) includes displaying 
by embedding the AV scene in at least part of the markup document 
scene. 

57. The method of claim 55, wherein step (c) includes displaying 
by using a picture in picture (PIP) technique for overlapping the AV 
scene on the markup document scene. 

58. The method of claim 55, wherein step (c) includes displaying 
by overlapping the AV scene and the markup document scene. 

59. The method of claim 55, wherein step (b) includes 

71 



BNSDOCID: <WO. 



.03036460A1_I_: 



WO 03/036460 



PCT/KR02/01977 



document scene , markup document window desi g nation information for 

designating a window in which the markup document is displayed on a 
screen of a display, AV trimming area designation information for 
designating an area to increase and reduce a desired portion of the AV 
5 scene, and AV scene window designation information for designating a 
window in which the AV scene is displayed on the screen of the display. 

64. The method of claim 61, wherein step (b) further includes 
interpreting a program being coded on the basis of an object having 

1 0 property variables for controlling the CSS and for controlling the CSS. 

65. The method of claim 64, wherein step (b13) further includes 
increasing/reducing an output AV scene by controlling a property for 
designating a display mode of the AV scene and a property for 

1 5 designating a trimming area of the AV scene to increase and reduce a 
desired portion of the AV scene according to a user's input by the 
program using an object source. 

66. The method of claim 64, wherein step (b1 3) further 

20 controlling a property for designating a trimming area of the synthesized 
scene, a property for designating a window area of the synthesized 
scene, and a property for designating a trimming area of the AV scene 
according to the screen display mode by the program using an object 
source in response to user's screen mode conversion, thereby 

25 reconstituting the aspect ratio of the scene. 

67. The method of claim 64, wherein step (b1 3) further includes 
controlling a property for designating a trimming area of the markup 
document scene, a property for designating a window area of the markup 

30 document scene, a property for designating a trimming area of the AV 
scene, and a property for designating a window area of the AV scene 
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s ynthesis information defined in a corres ponding st yle sheet if there is a 

style sheet in the markup document. 

70. The method of claim 69, wherein step (b3) comprises: 

5 (b31 ) determining whether a scene mode is changed by the user; 

(b32) if the scene mode is changed, transmitting scene mode 
change control information to the markup document and executing a 
script therefrom; and 

(b33) changing the output state of the markup document scene in 
10 response to scene mode change using screen display properties. 

71 . The method of claim 70, wherein step (b33) includes 

in an embedded mode of an interactive mode or a PIP mode, outputting 
the AV data at a first aspect ratio when the AV data having the first 
15 aspect ratio is displayed on a display having a second aspect ratio 
having a resolution lower than that of the first aspect ratio, and in a 
background mode of the interactive mode or video mode, outputting the 
AV data in a pan & scan or letterbox shape. 

20 72. The method of claim 69, wherein step (b) comprises: 

(b1 ) selecting a default style sheet in the reproducing apparatus 
on the basis of a set scene mode; 

(b2) reading scene synthesis information defined in the selected 
default style sheet including markup document window designation 

25 information for designating a window in which the markup document 
scene is displayed on the screen of the display, markup document 
trimming area designation information for designating a trimming area of 
the markup document scene, AV trimming area designation information 
for designating an area to increase and reduce a desired portion of the 

30 AV scene, and AV window area designation information for designating a 

75 



k- BNSDOCID: <WO 0303646QA1_L> 



WO 03/036460 



PCT/KR02/01977 



a decoder which decodes the AV data read by the reading unit 

and outputs an AV scene; and 

a controller which interprets the markup document read by the 
reading unit, outputs a markup document scene, interprets scene 
5 synthesis information which describes at least two display modes for 
displaying the markup document scene together the AV scene, and 
displays the AV scene and the markup document scene in one of the 
display modes according to the interpreted scene synthesis information. 

10 76. The apparatus of claim 75, wherein the scene synthesis 

information includes at least one of display mode designation 
information for designating a display mode of the AV scene, background 
color designation information for designating a background color of the 
scene, trimming area designation information for designating a trimming 

15 area of a scene in which the AV scene is synthesized with the markup 
document scene, window designation information for designating a 
window in which the synthesized scene is displayed on a screen of a 
display, and AV trimming area designation information for designating an 
area to increase and reduce a desired portion of the AV scene. 

20 

77. The apparatus of claim 76, wherein the display mode 
designation information indicates at least one of an embedded mode in 
which at least the AV scene is embedded in at least part of the markup 
document scene, a picture in picture (PIP) mode in which the AV scene 

25 is overlapped on the markup document scene, and a background mode 
in which the markup document scene is overlapped on the AV scene. 

78. The apparatus of claim 75, wherein the controller interprets 
a cascading style sheet (CSS) linked to or embedded in the markup 

30 document and interprets a program being coded on the basis of property 
variables for controlling the CSS and for controlling the CSS. 
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r color designation information for designating a background color of the 

scene, trimming area designation information for designating a trimming 
area of a scene in which the AV scene is synthesized with the markup 
document scene, window designation information for designating a 
5 window in which the synthesized scene is displayed on a screen of a 
display, and AV trimming area designation information for designating an 
area to increase and reduce a desired portion of the AV scene. 

83. The apparatus of claim 82, wherein the display mode 

10 designation information indicates at least one of an embedded mode in 
which at least the AV scene is embedded in at least part of the markup 
document scene, a picture in picture (PIP) mode in which the AV scene 
is overlapped on the markup document scene, and a background mode 
in which the markup document scene is overlapped on the AV scene. 

15 

84. The apparatus of claim 83, wherein the controller interprets 
a cascading style sheet (CSS) linked to or embedded in the markup 
document and interprets a program being coded on the basis of property 
variables for controlling the CSS and for controlling the CSS. 

20 

85. The apparatus of claim 84, wherein the controller controls a 
property for designating a trimming area of the synthesized scene, a 
property for designating a window area of the synthesized scene, and a 
property for designating a trimming area of the AV scene in the style 

25 sheet file interpreted according to a screen display property, by the 
program using an object source in response to user's scene mode 
change. 

86. The apparatus of claim 83, wherein the controller controls 
30 that in an embedded mode of an interactive mode or a PIP mode, when 

the AV data having a first aspect ratio is displayed on a display having a 
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trimming area designation information and window area designation 

information of the markup document scene and trimming area 
designation information and window area designation information of the 
AV scene, which are respectively defined, when the AV data having the 
5 first aspect ratio is displayed on a display having a second aspect ratio 
having a resolution lower than that of the first aspect ratio. 

90. The apparatus of claim 81, further comprising a blender 
which displays the AV scene and the markup document scene together 
10 by a display command according to the scene synthesis information 
interpreted by the controller. 
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FIG. 8 
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FIG. 15 
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FIG. 23 
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